IN THE CLAIMS 

Please cancel Claims 1 and 12, without prejudice or disclaimer of subject matter. 
Please amend Claims 2-4, 6, 7, 11, 13-18, 22, 23, 25, 27-31 and 33, and add new 
Claims 35-37, to read as follows. 

1 . (Canceled) 

2. (Currently Amended) Data processing apparatus according to claim [[1]] 
4, wherein said analysing means is adapted to identify the positions in accordance with pause 
criteria for the natural language input. 

3. (Currently Amended) Data processing apparatus according to claim [[1]] 
4 or claim 2^ wherein said generating means is adapted to add marker means to the identified 
positions in the grammar rules at which pauses can occur in the natural language input to 
generate the modified data structure. 



4. (Currently Amended) Data processing apparatus acco r ding to Claim 1 or 2 

comprising: 

receiving means for receiving a data structure defining grammar rules for 
recognition of a natural language input: 

analysing means for analysing the data structure to identify positions in the 
grammar rules at which pauses can occur in the natural language input: and 
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generating means for generating a modified data structure defining modified 
grammar rules for recognition of a natural language input with pauses therein, 

wherein said generating means is adapted to fragment the grammar rules in 
accordance with said identified positions to generate sub grammar rules to form said modified 
data structure. 

5. (Original) Data processing apparatus according to claim 4, wherein said 
generating means is adapted to form a hierarchical structure using said sub grammar rules to . 
form said modified data structure. 

6. (Currently Amended) Data processing apparatus according to Claim [[1]] 
4 or 2^ wherein said receiving means is adapted to receive a data structure defining grammar 
rules for use in speech recognition of a natural language speech input, and said generating means 
is adapted to generate said modified data structure defining modified grammar rules for speech 
recognition of a natural language speech input with pauses therein. 



7. (Currently Amended) Data processing apparatus according to Claim 1 o r 

2 comprising: 

receiving means for receiving a data structure defining grammar rules for 
recognition of a natural language input: 

analvsing means for analvsing the data structure to identify^ positions in the 
grammar rules at which pauses can occur in the natural language input: and 
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generating means for generating a modified data structure defining modified 
grammar rules for recognition of a natural language input with pauses therein, 

wherein said receiving means is adapted to receive said data structure defining 
grammar rules for recognition of a natural language input as a first modality input in conjunction 
with associated events in at least one further modality input, said data structure defining the 
association between events in each modality input, and events in said first modality input 
comprising units in the natural language. 

8. (Original) Data processing apparatus according to claim 7, wherein said 
analysing means is adapted to identify said positions in the grammar rules based on events in at 
least one said further modality input. 

9. (Original) Data processing apparatus comprising: 

receiving means for receiving a data structure defining grammar rules for 
recognition of a natural language input; 

analysing means for analysing the data structure to identify positions in the 
grammar rules at which events in at least one further modality input can occur in the natural 
language input as a first modality input; and 

generating means for generating a modified data structure defining modified 
grammar rules for recognition of a natural language input as the first modality input in 
conjunction with associated events in a said further modality input, said data structure defining 
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the association between events in each modality input, and events in said first modality input 
comprising units in the natural language. 



10. (Original) Data processing apparatus according to claim 9, wherein said 
analysing means is adapted to identify positions in the grammar rules at which pauses can occur 
in the natural language input based on events in at least one said further modality input. 

1 1 . (Currently Amended) Data processing apparatus according to claim [[,]] 9 
or 10, wherein said generating means is adapted to generate a further modified data structure 
defining said modified grammar rules and the relationships with events in the o r each further 
modality input. 

12. (Canceled) 

13. (Currently Amended) A data processing method according to claim [[12]] 
15 , wherein the analysing step includes identifying the positions in accordance with pause criteria 
for the natural language input. 

14. (Currently Amended) A data processing method according to claim [[12]] 
15 or claim 13, wherein the generating step includes adding marker means to said data structure 
to identify the positions in the grammar rules at which pauses can occur in the natural language 
input. 
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1 5 . (Currently Amended) A data processing method according to Claim 12 o r 
i3 comprising the steps of: 

receiving a data structure defining grammar rules for recognition of a natural 
language input: 

analysing the data structure to identify positions in the grammar rules at which 
pauses can occur in the natural language input: and 

generating a modified data structure defining modified grammar rules for 
recognition of a natural language input with pauses therein, 

wherein the generating step includes fragmenting the grammar rules in accordance 
with said identified positions to generate sub grammar rules to form said modified data structure. 

16. (Currently Amended) A data processing method according to Claim 15^ 
wherein the generating step includes forming a hierarchical structure using said sub grammar 
rules to form said modified data structure. 



1 7. (Currently Amended) A data processing method according to Claims 
[[12]] 15 or 13, wherein the receiving step comprises receiving a data structure defining grammar 
rules for use in speech recognition of a natural language speech input, and the generating step 
comprises generating a modified data structure defining modified grammar rules for speech 
recognition of a natural language speech input with pauses therein. 
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18. (Currently Amended) A data processing method according to Claim 12 or 
comprising the steps of: 

receiving a data structure defining grammar rules for recognition of a natural 
language input: 

analysing the data structure to identify positions in the grammar rules at which 
pauses can occur in the natural language input: and 

generating a modified data structure defining modified graromar rules for 
recognition of a natural language input with pauses therein, 

wherein the receiving step comprises receiving a data structure defining grammar 
rules for recognition of a natural language input as a first modality input in conjunction with 
associated events in at least one further modality input, said data structure defining the 
association between events in each modality input, events in said first modality input comprising 
units in the natural language. 

19. (Original) A data processing method according to claim 18, wherein the 
analysing step comprises identifying said positions in the grammar rules based on events in at 
least one said further modality input. 

20. (Original) A data processing method comprising; 

receiving a data structure defining grammar rules for recognition of a natural 
language input; 
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analysing the data structure to identify positions in the grammar rules at which 
events in at least one further modality input can occur in the natural language input as a first 
modality input; and 

generating a modified data structure defining modified grammar rules for 
recognition of a natural language input as the first modality input in conjimction with associated 
events in a said further modality input, said data structure defining the association between 
events in each modality input, and events in said first modality input comprising units in the 
natural language. 



21 . (Original) A data processing method according to claim 20, wherein the 
analysing step comprises identifying positions in the grammar rules at which pauses can occur in 
the natural language input based on events in at least one said further modality input. 

22. (Currently Amended) A data processing method according to claim 21, 
wherein the generating step includes generating a further modified data structure defining said 
modified grammar rules and the relationships with events in thc^ each further modality input. 

23. (Currently Amended) Apparatus for generating data in a computer usable 
form, the apparatus comprising: 

receiving means for receiving a natural language input with a number of pauses 

therein; and 
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recognition means for recognising said natural language input using the modified 
data structure generated using the method of any one of Claims [[12]] 15, 13, 20[[,]] or 21 to 
generate data in computer usable form. 



24. (Original) Apparatus according to claim 23, wherein said recognising 
means comprises speech recognition means for recognising a natural language speech input. 

25. (Currently Amended) A method of generating data in a computer usable 
form, the method comprising; 

receiving a natural language input with a number of pauses therein; and 
recognising said natural language input using the modified data structure 
generated using the method of Claim 20 to generate data in computer usable form. 

26. (Original) A method according to claim 25, wherein the recognising step 
comprises speech recognition of a natural language speech input. 

27. (Currently Amended) Apparatus for generating data in a computer usable 
form, the apparatus comprising: 

first modality receiving data generated for a natural language input by the 
apparatus of claim 23, said data comprising recognised units of the natural language and 
comprising data of a first modality input; 
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further modality receiving means for receiving data identifying events in at least 
one further modality input; 

data structure receiving means for receiving a further modified data structure 
defining modified grammar rules and the relationships with events in the o r each further 
modality, said further modified data structure having been generated using the method of claim 
18; 

analysing means for analysing the first modality input data and the o r each further 
modality input data to determine if they match with any said modified grammar rule and related 
events in the o r each further modality; and 

generating means for generating computer usable data in dependence upon said 
analysis by said analysing means. 

28. (Currently Amended) Apparatus according to claim 27, wherein said first 
modality receiving means is adapted to receive recognition data comprising an ordered list of 
likely natural language units to accompany the most likely natural language for each natural 
language unit recognised, and said analysing means is adapted to use said ordered list when the 
most likely natural language units do not result in a match with any modified grammar rule and 
related events in thc-or each further modality. 

29. (Currently Amended) Apparatus according to claim 27^ wherein said first 
modality receiving means is adapted to receive speech recognition data. 
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30. (Currently Amended) A method of generating data in a computer usable 
form, the method comprising: 

a first receiving step of receiving data generated for a natural language input by 
the method of claim 25, said data comprising recognised units of the natural language, and 
comprising data of a first modality input; 

a second receiving step of receiving data identifying events in at least one further 
modality input; 

a third receiving step of receiving a further modified data structure defining 
modified grammar rules and the relationship with events in thc-or each further modality, said 
further modified data structure having been generated du r ing using the method of claim 22; 

analysing the first modality input data and thc-or each further modality input data 
to determine if they match with any said modified grammar rule and related events in the o r each 
further modality; and 

generating computer usable data in dependence upon said analysis. 

3 1 . (Currently Amended) A method according to claim 30^ wherein the first 
receiving step comprises receiving recognition data comprising an ordered list of likely natural 
language units to accompany the most likely natural language unit for each natural language unit 
recognised, and the analysis step includes using the ordered list when the most likely natural 
language units do not result in a match with any modified grammar rule and related events in 
the or each further modality. 
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32. (Previously Presented) A method according to claim 31, wherein the first 
receiving step receives speech recognition data. 



33. (Currently Amended) Processor implementable instructions for 
controlling a processor to carry out the method of any one of claims [[12]] 15, 20, 21ror 22, or 
25[[,]]. 

34. (Original) A carrier medium for carrying the processor implementable 
instructions according to claim 33. 

35. (New) Data processing apparatus comprising: 

a receiver operable to receive a data structure defining grammar rules for 
recognition of a natural language input; 

an analyser operable to analyse the data structure to identify positions in the 
grammar rules at which pauses can occur in the natural language input; and 

a generator operable to generate a modified data structure defining modified 
grammar rules for recognition of a natural language input with pauses therein, 

wherein said generator is operable to fragment the grammar rules in accordance 
with said identified positions to generate sub grammar rules to form said modified data structure. 

36. (New) Data processing apparatus comprising: 
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a receiver operable to receive a data structure defining grammar rules for 
recognition of a natural language input; 

an analyser operable to analyse the data structure to identify positions in the 
grammar rules at which pauses can occur in the natural language input; and 

a generator operable to generate a modified data structure defining modified 
grammar rules for recognition of a natural language input with pauses therein, 

wherein said receiver is operable to receive said data structure defining grammar 
rules for recognition of a natural language input as a first modality input in conjunction with 
associated events in at least one fiirther modality input, said data structure defining the 
association between events in each modality input, and events in said first modality input 
comprising units in the natural language. 

37. (New) Data processing apparatus comprising: 

a receiver operable to receive a data structure defining grammar rules for 
recognition of a natural language input; 

an analyser operable to analyse the data structure to identify positions in the 
grammar rules at which events in at least one further modality input can occur in the natural 
language input as a first modality input; and 

a generator operable to generate a modified data structure defining modified 
grammar rules for recognition of a natural language input as the first modality input in 
conjunction with associated events in a said further modality input, said data structure defining 
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the association between events in each modality input, and events in said first modaUty input 
comprising units in the natural language. 
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